59 research outputs found
GANN: Graph Alignment Neural Network for Semi-Supervised Learning
Graph neural networks (GNNs) have been widely investigated in the field of
semi-supervised graph machine learning. Most methods fail to exploit adequate
graph information when labeled data is limited, leading to the problem of
oversmoothing. To overcome this issue, we propose the Graph Alignment Neural
Network (GANN), a simple and effective graph neural architecture. A unique
learning algorithm with three alignment rules is proposed to thoroughly explore
hidden information for insufficient labels. Firstly, to better investigate
attribute specifics, we suggest the feature alignment rule to align the inner
product of both the attribute and embedding matrices. Secondly, to properly
utilize the higher-order neighbor information, we propose the cluster center
alignment rule, which involves aligning the inner product of the cluster center
matrix with the unit matrix. Finally, to get reliable prediction results with
few labels, we establish the minimum entropy alignment rule by lining up the
prediction probability matrix with its sharpened result. Extensive studies on
graph benchmark datasets demonstrate that GANN can achieve considerable
benefits in semi-supervised node classification and outperform state-of-the-art
competitors
Revisiting Initializing Then Refining: An Incomplete and Missing Graph Imputation Network
With the development of various applications, such as social networks and
knowledge graphs, graph data has been ubiquitous in the real world.
Unfortunately, graphs usually suffer from being absent due to
privacy-protecting policies or copyright restrictions during data collection.
The absence of graph data can be roughly categorized into attribute-incomplete
and attribute-missing circumstances. Specifically, attribute-incomplete
indicates that a part of the attribute vectors of all nodes are incomplete,
while attribute-missing indicates that the whole attribute vectors of partial
nodes are missing. Although many efforts have been devoted, none of them is
custom-designed for a common situation where both types of graph data absence
exist simultaneously. To fill this gap, we develop a novel network termed
Revisiting Initializing Then Refining (RITR), where we complete both
attribute-incomplete and attribute-missing samples under the guidance of a
novel initializing-then-refining imputation criterion. Specifically, to
complete attribute-incomplete samples, we first initialize the incomplete
attributes using Gaussian noise before network learning, and then introduce a
structure-attribute consistency constraint to refine incomplete values by
approximating a structure-attribute correlation matrix to a high-order
structural matrix. To complete attribute-missing samples, we first adopt
structure embeddings of attribute-missing samples as the embedding
initialization, and then refine these initial values by adaptively aggregating
the reliable information of attribute-incomplete samples according to a dynamic
affinity structure. To the best of our knowledge, this newly designed method is
the first unsupervised framework dedicated to handling hybrid-absent graphs.
Extensive experiments on four datasets have verified that our methods
consistently outperform existing state-of-the-art competitors
Attribute Graph Clustering via Learnable Augmentation
Contrastive deep graph clustering (CDGC) utilizes contrastive learning to
group nodes into different clusters. Better augmentation techniques benefit the
quality of the contrastive samples, thus being one of key factors to improve
performance. However, the augmentation samples in existing methods are always
predefined by human experiences, and agnostic from the downstream task
clustering, thus leading to high human resource costs and poor performance. To
this end, we propose an Attribute Graph Clustering method via Learnable
Augmentation (\textbf{AGCLA}), which introduces learnable augmentors for
high-quality and suitable augmented samples for CDGC. Specifically, we design
two learnable augmentors for attribute and structure information, respectively.
Besides, two refinement matrices, including the high-confidence pseudo-label
matrix and the cross-view sample similarity matrix, are generated to improve
the reliability of the learned affinity matrix. During the training procedure,
we notice that there exist differences between the optimization goals for
training learnable augmentors and contrastive learning networks. In other
words, we should both guarantee the consistency of the embeddings as well as
the diversity of the augmented samples. Thus, an adversarial learning mechanism
is designed in our method. Moreover, a two-stage training strategy is leveraged
for the high-confidence refinement matrices. Extensive experimental results
demonstrate the effectiveness of AGCLA on six benchmark datasets
Large Language Models are Zero Shot Hypothesis Proposers
Significant scientific discoveries have driven the progress of human
civilisation. The explosion of scientific literature and data has created
information barriers across disciplines that have slowed the pace of scientific
discovery. Large Language Models (LLMs) hold a wealth of global and
interdisciplinary knowledge that promises to break down these information
barriers and foster a new wave of scientific discovery. However, the potential
of LLMs for scientific discovery has not been formally explored. In this paper,
we start from investigating whether LLMs can propose scientific hypotheses. To
this end, we construct a dataset consist of background knowledge and hypothesis
pairs from biomedical literature. The dataset is divided into training, seen,
and unseen test sets based on the publication date to control visibility. We
subsequently evaluate the hypothesis generation capabilities of various
top-tier instructed models in zero-shot, few-shot, and fine-tuning settings,
including both closed and open-source LLMs. Additionally, we introduce an
LLM-based multi-agent cooperative framework with different role designs and
external tools to enhance the capabilities related to generating hypotheses. We
also design four metrics through a comprehensive review to evaluate the
generated hypotheses for both ChatGPT-based and human evaluations. Through
experiments and analyses, we arrive at the following findings: 1) LLMs
surprisingly generate untrained yet validated hypotheses from testing
literature. 2) Increasing uncertainty facilitates candidate generation,
potentially enhancing zero-shot hypothesis generation capabilities. These
findings strongly support the potential of LLMs as catalysts for new scientific
discoveries and guide further exploration.Comment: Instruction Workshop @ NeurIPS 202
Dink-Net: Neural Clustering on Large Graphs
Deep graph clustering, which aims to group the nodes of a graph into disjoint
clusters with deep neural networks, has achieved promising progress in recent
years. However, the existing methods fail to scale to the large graph with
million nodes. To solve this problem, a scalable deep graph clustering method
(Dink-Net) is proposed with the idea of dilation and shrink. Firstly, by
discriminating nodes, whether being corrupted by augmentations, representations
are learned in a self-supervised manner. Meanwhile, the cluster centres are
initialized as learnable neural parameters. Subsequently, the clustering
distribution is optimized by minimizing the proposed cluster dilation loss and
cluster shrink loss in an adversarial manner. By these settings, we unify the
two-step clustering, i.e., representation learning and clustering optimization,
into an end-to-end framework, guiding the network to learn clustering-friendly
features. Besides, Dink-Net scales well to large graphs since the designed loss
functions adopt the mini-batch data to optimize the clustering distribution
even without performance drops. Both experimental results and theoretical
analyses demonstrate the superiority of our method. Compared to the runner-up,
Dink-Net achieves 9.62% NMI improvement on the ogbn-papers100M dataset with 111
million nodes and 1.6 billion edges. The source code is released at
https://github.com/yueliu1999/Dink-Net. Besides, a collection (papers, codes,
and datasets) of deep graph clustering is shared at
https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering.Comment: 19 pages, 5 figure
Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment
Temporal graph learning aims to generate high-quality representations for
graph-based tasks along with dynamic information, which has recently drawn
increasing attention. Unlike the static graph, a temporal graph is usually
organized in the form of node interaction sequences over continuous time
instead of an adjacency matrix. Most temporal graph learning methods model
current interactions by combining historical information over time. However,
such methods merely consider the first-order temporal information while
ignoring the important high-order structural information, leading to
sub-optimal performance. To solve this issue, by extracting both temporal and
structural information to learn more informative node representations, we
propose a self-supervised method termed S2T for temporal graph learning. Note
that the first-order temporal information and the high-order structural
information are combined in different ways by the initial node representations
to calculate two conditional intensities, respectively. Then the alignment loss
is introduced to optimize the node representations to be more informative by
narrowing the gap between the two intensities. Concretely, besides modeling
temporal information using historical neighbor sequences, we further consider
the structural information from both local and global levels. At the local
level, we generate structural intensity by aggregating features from the
high-order neighbor sequences. At the global level, a global representation is
generated based on all nodes to adjust the structural intensity according to
the active statuses on different nodes. Extensive experiments demonstrate that
the proposed method S2T achieves at most 10.13% performance improvement
compared with the state-of-the-art competitors on several datasets
Reinforcement Graph Clustering with Unknown Cluster Number
Deep graph clustering, which aims to group nodes into disjoint clusters by
neural networks in an unsupervised manner, has attracted great attention in
recent years. Although the performance has been largely improved, the excellent
performance of the existing methods heavily relies on an accurately predefined
cluster number, which is not always available in the real-world scenario. To
enable the deep graph clustering algorithms to work without the guidance of the
predefined cluster number, we propose a new deep graph clustering method termed
Reinforcement Graph Clustering (RGC). In our proposed method, cluster number
determination and unsupervised representation learning are unified into a
uniform framework by the reinforcement learning mechanism. Concretely, the
discriminative node representations are first learned with the contrastive
pretext task. Then, to capture the clustering state accurately with both local
and global information in the graph, both node and cluster states are
considered. Subsequently, at each state, the qualities of different cluster
numbers are evaluated by the quality network, and the greedy action is executed
to determine the cluster number. In order to conduct feedback actions, the
clustering-oriented reward function is proposed to enhance the cohesion of the
same clusters and separate the different clusters. Extensive experiments
demonstrate the effectiveness and efficiency of our proposed method. The source
code of RGC is shared at https://github.com/yueliu1999/RGC and a collection
(papers, codes and, datasets) of deep graph clustering is shared at
https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering on Github
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal
Knowledge graph reasoning (KGR), aiming to deduce new facts from existing
facts based on mined logic rules underlying knowledge graphs (KGs), has become
a fast-growing research direction. It has been proven to significantly benefit
the usage of KGs in many AI applications, such as question answering,
recommendation systems, and etc. According to the graph types, existing KGR
models can be roughly divided into three categories, i.e., static models,
temporal models, and multi-modal models. Early works in this domain mainly
focus on static KGR, and recent works try to leverage the temporal and
multi-modal information, which are more practical and closer to real-world.
However, no survey papers and open-source repositories comprehensively
summarize and discuss models in this important direction. To fill the gap, we
conduct a first survey for knowledge graph reasoning tracing from static to
temporal and then to multi-modal KGs. Concretely, the models are reviewed based
on bi-level taxonomy, i.e., top-level (graph types) and base-level (techniques
and scenarios). Besides, the performances, as well as datasets, are summarized
and presented. Moreover, we point out the challenges and potential
opportunities to enlighten the readers. The corresponding open-source
repository is shared on GitHub
https://github.com/LIANGKE23/Awesome-Knowledge-Graph-Reasoning.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
- …